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Abstract 

■ We present a new approach to description of national economies. For this we use 
i the state space viewpoint, which is used mostly in the theory of dynamical systems and 

1 ^ I in the control theory. Gross domestic product, inflation, and unemployment rates are 

' taken as state variables. We demonstrate that for the considered period of time the phase 

trajectory of each of the V4 countries (Slovak Republic, Czech Republic, Hungary, and 
Poland) lies approximately in one plane, so that the economic development of each country 
' can be assocated with a corresponding plane in the state space. The suggested approach 

. opens a way to a new set of economic indicators (for example, normal vectors of national 

QQ I economies, various plane slopes, 2D angles between the planes corresponding to different 

■ economies, etc.). 

. The tool used for computations is orthogonal regression (alias orthogonal distance 

\^ [ regression, alias total least squares method), and we also give general arguments for using 

' orthogonal regression instead of the classical regression based on the least squares method. 

A MATLAB routine for fitting 3D data to lines and planes in 3D is provided. 



Keywords: orthogonal regression, orthogonal distance regression, total least squares 
method, state space description, phase trajectory, national economy. 



^ 1 Introduction 



In this article we consider two different versions of the least squares method - the classical one 
and the so called orthogonal regression. We incline to using the orthogonal regression, and give 
several arguments in favour of this. The main reason is the fact that orthogonal regression is 
a suitable tool for fitting lines and surfaces in multidimentional space, while even the use of 
classical regression in 2D is not so natural. 
The article is organized as follows. 

First, we briefly recall the history of the classical least squares method to remind that the 
justification of its applications is unclear from its very beginning. 

Second, we take a critical look at the traditional interpretation of the classical least squares 
method and conclude that it is absolutely artificial and meaningless. Changing a viewpoint 
from "squares" to "circles", we find a natural geometric interpretation for quantities to be 
minimized, and this change leads to the method of orthogonal regression. 
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After that, we provide several general arguments in favour of using orthogonal regression. 

We provide an easy example - "The Flight of a Bumblebee" - in order to illustrate how the 
orthogonal regression method can help in describing dependences in multidimentional space. 

Then we apply the orthogonal regression method to describing the national ecomonies of the 
countries of the V4 group in the state space of three variables - gross domestic product, inflation, 
and unemployment. The development of the national economies under study is described by 
their trajectories in the chosen state space. We discovered that for each particular country the 
trajectory of its national economy lies approximately in one plane, so the normal vector of this 
plane and one of its points (for example, the one coinciding with the data centroid) can be 
associated with a particular economy. 

In the conclusion we discuss some new possibilities that our approach opens in the field of 
description of complex dynamic economic systems such as national economies. 

2 The history of the least squares method 

One can hardly find another topic, which is so famous and which has such misterious and 
unclear history as the least squares method. 

The controversal circumstances around the the appearance of the method of least squares 
were investigated by many authors. According to Celmins p], it might be summarized as 
follows. 

In 1805 Legendre published "Nouvelles methodes pour la determination des cometes", in 
which he introduced the method of least squares and gave it this famous name. 

In 1809, Gauss published the book "Theoria motus corporum coelestium in sectionibus coni- 
cis solem ambientium" jSE], where he discussed the method of least squares and, mentioning 
Legendre's work, stated that he himself had used the method since 1795. 

Legendre felt offended by Gauss's statement. In a letter to Gauss about his new book Leg- 
endre wrote that claims of priority should not be made without proof by previous publications. 
Gauss did not have such a publication, but he stated that he was convinced that the idea of 
least squares method is so simple that many people must have used the method even before 
him. 

Later Gauss tried to prove his claim but had only little success. His own computational 
notes were - as he said - lost. His colleagues apparently did not remember discussions with him 
or did not want to be involved in the controversy. As Celmins mentions [T], only the astronomer 
Olbers included in a paper in 1816 a footnote asserting that Gauss had shown him the method 
of least squares in 1802, and Bessel published a similar note in a report in 1832. 

Later Gauss gave up the search but did not retract his claim. In 1820 Legendre published 
a supplement to his 1805 memoir with an appendix where he publicly attacked Gauss's claims 
of priority. The controversy continued, and in 1831 H. C. Schumacher (cf. Celmins [Ijj) wrote 
to Gauss about a publication published in 1799 that contained data and adjustment results by 
Gauss. Schumacher suggested repeating the calculations and thereby demonstrating that the 
method of least squares was indeed used by Gauss in 1799. Gauss's answer was that he was 
well aware of the data but would not permit a recalculation, since such attempts would only 
suggest that he could not be trusted. 

After describing the above circumstances, Celmins tried to repeat Gauss's calculations. 
Celmins's main conclusion was that Gausss results are not consistent with any obvious and 
reasonable adjustment of observational errors nor with a least-square adjustment in the param- 
eter space. 
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(a) Fitting data to a straight line 



(b) Geometric interpretation(?) of the classical 
LSM 



Figure 1: Fitting by the classical least squares method (LSM) 



3 Classical LSM 



The well known least squares method is a mathematical procedure for finding the best-fitting 
curve to a given set of points by minimizing the sum of the squares of the offsets ( "the residuals" ) 
of the points from the curve. 

The classical least squares fitting consists in minimizing the sum of the squares of the vertical 
deviations of a set of data points 



from a chosen function /. 

For a simple illustration, let us recall the classical linear regression problem, in which we 
have to fit the set of data by a straight line in 2D plane. This situation is shown in Fig. |l(a)[ 

One may use the edge of a transparent ruler or a tightly stretched black string to get a line 
which seems to fit the data points. Mathematically, what we have to do is to determine the 
parameters k and b of the equation of a straight line: 



where x - the variable considered as independent; y - the variable considered as dependent 
on X] k, b - constants, often called parameters, to be determined so that the line fits the data 
optimally (in some sense...). 

To find a way to calculate the parameters, let us return to the simplest case, equation Q 
and Fig. |l(a)[ and let us agree to choose k and b so as to minimize the sum of the squares 
of the errors. Fig. |l(b)| shows these squares graphically. We twist and push the line through 
these points until the sum of the areas of the squares is the smallest. The natural question is: 
"Why the squares? Why not just get the smallest sum of distances of the data points from the 
line?" The only real explanation is that it is easy to compute k and b to minimize the sum 
of squares off offsets of y (vertical offsets), but it is quite difficult to minimize (using analytic 
derivations) the sum of distances of the data points from the line, and it is really this ease that 
is responsible for the generally accepted preference for the squares of offsets of y. 

In fact, we must always keep in mind that the least squares approach is basically a "last 
resource" tool that is used for obtaining at least some mathematical model for a process under 



E = Y^lVi - f{xi,ai,a2, . . . 



an)? 




y = kx + b 
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(a) The case of classical least squares fitting 



(b) The case of orthogonal distance fitting 



Figure 2: "Least circles" viewpoint 



study, when obtaining a better model by analytical derivations is impossible or extremely time 
and/or effort consuming. As such, it is based on several presumptions, which are based only 
on some intuition or prejustice; among them we must mention postulating the role of variables 
("independent/dependent") and the type of dependance between them (linear, polynomial, 
exponential, logarithmic, etc.) 

4 The method of least circles?! 



Looking at the geometric "interpretation" of the least squares method shown in Fig. |l(b)] which 
so often appears in numerous textbooks and lectures, we can conclude that it is absolutely 
artificial and does not contain any sign of mathematical beauty. The picture shown in Fig. |l(b)| 
again provokes the question: "Why the squares?" 

To change a viewpoint, let us note that the criterion can be painlessly replaced with 



Indeed, multiplication by a non-zero number (3 does not affect the point of minimum. Only 
the minimum value of the criterion function {E) will be multiplied by /5 - but this value itself 
is not the subject of interest, since we look for the values of ai, a2, ... 

Taking (3 = n, we obtain 



Geometrically, the formula (jH) means the sum of areas of the circles shown in Fig. |2(a)| 
The radii of the circles in Fig. |2(a)| are the vertical offsets of yi from the fitting line. Each of 
those circles has two points of intersection with the line. It is clear, that one cannot consider 
this picture as elegant. Changing the radii slightly, one can preserve n pairs of intersection 
of the circles and the line, so one can draw an infinite number of pictures lloking similar to 
Fig. |2(a)j But Fig. |2(a)| is just a reformulation of the standard geometric "illustration" of the 
least squares method (recall Fig. |l(b)| ). Instead of the "least squares method" we now deal 
with the "least circles method". But the circles shown in Fig. |2(a)|are clearly not the best. 



E = PY^lVi - f{xi,ai,a2, . . . 




(3) 



E = ^AVi - f{xi,ai,a2, . . . 



an)? 



(4) 



Figure 3: The case of non-carthesian coordinates. 



However, the circles shown in Fig. |2(b)| are really optimal: the fitting line is a tangent line 



to all circles. The radii of the circles in Fig. |2(b)| are equal to minimal distances between the 
points {xi,yi) and the fitting line, and this guarantees the unique picture. 
The criterion to minimize in this case is 



which is up to a constant multiplier n the formula known under the name of orthogonal re- 
gression or total least squares jSl IH El CHI • Here d({xi, yi), denotes the distance between the 
point {xi,yi) and the fitting line /. 

5 Arguments for orthogonal regression approach 

There are numerous works and significant actvities devoted to the theory of the total least 
squares method [12]. To support its wider applications, we would like to list the following 
arguments in favour of the orthogonal distance (or total least squares) fitting. 

1. The shortest (orthogonal) distance is the most natural viewpoint on any fitting. 

The section 0] and the comparison of Fig. |2(^ and Fig. |2(b)1 give a clear visual evidence 



2. The sum of orthogonal distances is invariant with respect to the choice of the system of 
coordinates. 

This is obvious, since the distance is independent of the choice of the coordinate system. In 
addition, let us imagine that we replace rectangular coordinates with non-rectangular. For 
example, in Fig. Elan affine system with the angle of 60 degrees between the axes is shown. 
In this picture the squares of Ayi have even less meaning than in Fig. |l(b)| However, the 
orthogonal distances, which are also shown in Fig. still have solid interpretation and 
can be used for fitting. 

3. There are no conjugate regression lines, which appear after swapping x and y, because in 
the case of orthogonal regression the fitting y = f{x) gives exactly the same line as the 
fitting X = f~^{y). 

Suppose one wants to find the dependance between the height (x) and the weight {y) 
of people. The dependance is presumed to be linear (a straight line) described by equa- 
tion Q. After determining k and h, this relaitonship can be used for estimating the 
weight of a person of a given height. 




(5) 



for this. 
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Table 1: Data for the Nievergelt's example. 



X 


1 
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4 5 


7 


y 
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6 8 


5 
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Figure 4: Nievergelt's example: classical regression versus orthogonal regression. 



However, the vewpoint can be inverted: for a given weight, estimate the height of a 
person. If one already has the equation Q, then the solution should normally be 

1 h 

However, the classical regression approach leads to a different result expressed by a con- 
jugate regression line. 

In Fig. m a solution to the Nievergelt's easily understandable example is shown. 
To make the situation more obvious, we added one to the ordinates in the Nievergelt's 
example, so we used the data shown in Table [U The classical least squares regression y 
versus x gives the regression line y = 0.45x + 3.2. After swapping x and y, classical least 
squares regression gives the conjugate regression line x = 0.45?/ + 1.75. However, in the 
case of the orthogonal distance regression we obtain the same line y = x + 1 {x = y — 1) 
independently on the order of x and y. 

It is worth noting that all three lines run through the centroid, and that the orthogo- 
nal distance regression line is located between the "scissors" formed by the conjugate 
regression lines obtained by the classical least squares regression. 

4. There are no problems with causality (normally, determination of what is an independent 
variable and what is a dependent variable is simply unclear or even impossible; this is 
always postulated) . 

In all textbooks on statistics discussing the classical regression analysis it is always un- 
derlined that that the choice of what is the "independent" variable and what if the 
"dependent" variable is extremely important. However, in many cases it is not so easy to 
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(6) 



make a decision what is what, and justification of such a decision is based only on some 
subjective judgement or on a prejustice. We already mentioned the relationship between 
the weight and the height of people. It is obvious that any of these characteristics can be 
taken as an independent one. 

5. Implementation of the orthogonal fitting does not depend on the number of dimensions. 

To realize this, just recall the square of the distance between the two points P{xi, X2, ■ ■ ■ , x„) 
and Q{yi,y2, ■ ■ ■ , Vn) in ri-dimensional orthogonal coordinate system is 



6 State space description of national economies 

In everyday professional and non-professional communication one can frequently hear the word 
"state of economy" . This expression can be given an exact meaning by adopting the tools that 
are available in the theory of dynamical systems and in automatic control. Namely, we will use 
the technique called state space description and phase trajectories. 

Suppose we have a dynamical process (that is, the process deveoping in time), which is 
characterized by three quantities x{t), y{t), z{t) (of course, number or dimensions can be 
different, not necessarily 3). Chosing x, y, and z as the three coordinates, we can assign a 
point {x{t),y(t), z(t)) to each value of t - that is, we assign a point in a 3D space to a state of 
the considered process at time t. The variables x, y, z are called the state variables. The line 
formed by the points {x{t),y{t), z{t)) when t takes on the values from a given interval (usually 
[0,T] for some finite T, or [0, oo)) is called the phase trajectory of the process. 

As state variables for this study, we selected those which are standard: gross domestic 
product (GDP), inflation, and unemployment. In the subsequent sections, we demonstrate the 
advantages of the state space description of national economies and possible ways for further 
developments and scientific investigations. 



7 Example: "The Flight of the Bumblebee" - fitting 3D 
data by a straight line in 3D 



Before continuing with real data for the countries of the V4 group, let us consider a simple 
general example. Suppose a bumblebee is flying from point A to point B. In an ideal case, 
the trajectory of its flight would be a straight line. However, for the reasons which are mostly 
unknown to us, we observe the deviations which we (due to lack of knowledge, or due to lack 
of a full and precise model of the bumblebee's motion) consider as random. 

Having a set of positions of the bumblebee (depicted as discrete points in Fig. E)), we can 
try to make a conclusion about the main direction of its flight. The simplest model is a straight 
line. The only way to determine the parameters of the equations of a fitting straight line in 3D 
space is using the method of orthogonal regression (total least squares). The result is shown in 
Fig. El as a straight line. 

Of course, the general trajectory of a bumblebee can be more complex than a straight line. 
Although in the case of orthogonal regression we do not need to postulate the roles of variables, 
we still need to postulate the structure of the model - we have to decide which line is more 
"promising" for modeling the bumblebee's flight - it could be a straight line, or a conic section, 
or some spiral, or something else. 



n 




(7) 



i=l 
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Figure 5: The Fhght of the Bumblebee. (Inspired by a figure from [^). 

The "bumblebee fiight" problem can be considered also as a problem of determining the 
direction of the fiight of an unknown aircraft, or determining the state of economy at some time 
instant in the future based on the observations in the past. 

8 The case of the Central European countries of the V4 
group 

The traditional approach to visualization of the change of GDP, infiation, and unemployment 
uses 2D plots - see Fig. |6(a)[ Fig. |6(b)[ and Fig. |6(c)} Although such visualization provides 
partial information about each particular aspect of a particular economy, and even allows 
comparisons of a given particular indicator for several economies, it does not provide any 
information about the economy as a whole. 

This is why we plot the data for each country of the V4 group - Slovak Republic (SK), Czech 
Republic (CZ), Hungary (HU), and Poland (PL) - in the state space with three coordinates: 
GDP, infiation, and unemployment. We use the data for the period from 1994 to 2000 from 
the MESA 10 report [Zj. 

One can see that, in conrast with the traditional 2D plots, the phase trajectories shown in 
Figs. |7(a)}]7(d) I nicely show how the national economies developed in time. 

The most interesting observation regarding these phase trajectories is that for each partic- 
ular country its phase trajectory lies approximately in one plane. This observation indicates 
that we can associated such planes in state space with particular economies, and that the global 
properties of each particular national economy as a whole are described by the associated plane 
(or, in other word, by its normal vector and the data centroid, which is a point belonging to 
the plane). 

The values of normal vectors and centroids of the planes describing national economies of 
the V4 countries are listed in Table El As it is well known from the analytic geometry, each 
plane is uniquely described by these data. In the last column the values of the total error of 
approximation are given. 

It is worth mentioning that the fitting of the state space data points to planes was done 
using total least squares (TLS) method, implemented in the form of the principal component 
analysis [Tn|, although in many cases the SVD decomposition approach is used fH HHj. The 
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Table 2: Economic indicators of the V4 countries, 1994-2000 



1994 1995 1996 1997 1998 1999 2000 

CZ 3^2 2J 3^5 5^2 7^5 9A STT" 

HU 11.2 10.5 9.2 7.7 7 6.5 6.5 

PL 16.0 14.9 13.5 10.5 10.4 13.0 13.5 

SK 13.7 13.1 11.3 11.8 12.5 16.2 18.5 



(a) Unemployment in the V4 countries - in percents 



1994 1995 1996 1997 1998 1999 2000 

CZ 2.2 5.9 4.8 -0.1 -2.2 -0.2 2.5 

HU 2.9 1.5 1.3 4.4 5.1 4.5 5.6 

PL 5.2 7 6.0 6.8 4.8 4.1 5.0 

SK 4.8 6.7 6.2 6.2 4.1 1.9 2.0 



(b) GDP of the V4 countries - change in percents 



1994 1995 1996 1997 1998 1999 2000 



CZ 


10 


9.1 


8.8 


8.5 


10.7 


2.1 


4.1 


HU 


18.8 


28.2 


23.6 


18.3 


14.3 


10 


9.3 


PL 


33.2 


28 


19.9 


14.8 


11.6 


7.3 


9.9 


SK 


13.4 


9.9 


5.8 


6.1 


6.7 


10.6 


11.5 



(c) Inflation in the V4 countries - in percents 



description of our function for MATLAB is given in the Appendix. 



9 Conclusions 

In this article we presented a new approach to description of national economies. This approach 
is based on the state space viewpoint, which is used mostly in the theory of dynamical systems 
and in the control theory. Gross domestic product, inflation, and unemployment rates were 
taken as state variables. We demonstrated that for the considered period of time the phase 
trajectory of each of the V4 countries (Slovak Repubhc, Czech Republic, Hungary, and Poland) 
lies approximately in one plane, so that the economic development of each country can be 
assocated with a corresponding plane in the state space. 




Figure 6: Classical 2D visualization of economic indicators of the V4 countries. 
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(a) Czech Republic 



(b) Hungary 




(c) Poland (d) Slovak Republic 

Figure 7: Phase trajectories of the national economies of the V4 countries in state space. 



(a) Czech Republic 



(b) Hungary 



(c) Poland 



(d) Slovak Republic 



Figure 8: Planes of the national economies of the V4 countries in state space. 
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Table 3: Normal vectors, centroids, and errors for the planes corresponding to the countries of 
the V4 group. 



Country 


Normal vector 


Centroid 


Error 


SK 


(0.6704, 0.7195, -0.1811) 


(13.8714, 4.5571, 9.1429) 


4.2633 


PL 


(-0.4083, -0.9059, 0.1123) 


(13.1143, 5.5571, 17.8143) 


4.3106 


CZ 


(0.7632, 0.4525, 0.4612) 


(5.7714, 1.8429, 7.6143) 


4.6111 


HU 


(0.7362, 0.6745, -0.0545) 


(8.3714, 3.6143, 16.0714) 


3.7431 



The suggested approach opens a way to a new set of economic indicators. Among possible 
indicators of economies we can mention, for example, normal vectors of national economies, 
various plane slopes, 2D angles between the planes corresponding to different economies, etc. 

The tool used for computations is orthogonal regression (alias orthogonal distance regression, 
alias total least squares method), and we also gave general arguments for using orthogonal 
regression instead of the classical regression based on the least squares method. 
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A Brief description of the MATLAB routine 

We created a MATLAB routine called f it_3D_data for performing computations used in our 
article. Since our routine can be used in many other investigations utilizing the orthogonal 
regression (total least squares) method, we published it at MATLAB File Exchange. The URI 
for download is: 

http: //www. mathworks . com/matlabcentral/f ileexchange/ 
loadFile.do?objectId=12395&objectType=f ile 
The description of f it_3D_data and its parameters is given below. 



function [Err,N,P] = f it_3D_data(XData, YData, ZData, 

geometry, visualization, sod) 

7. 

y„ [Err, N, P] = f it_3D_data(XData, YData, ZData, 

7o geometry, visualization, sod) 

7. 

7o Orthogonal Linear Regression in 3D-space 
7o by using Principal Components Analysis 
7. 

7o This is a wrapper function to some pieces of the code from 
7o the Statistics Toolbox demo titled "Fitting an Orthogonal 

% Regression Using Principal Components Analysis" 
'L (http:/ /www. mathworks . com/products/statistics/ 

'L demos . html?f ile=/products/demos/shipping/ stats/orthoregdemo . html) , 

7o which is Copyright by the MathWorks, Inc. 

7. 
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% Input parameters : 

% - XData: input data block — x: axis 

°L - YData: input data block — y: axis 

% - ZData: input data block — z: axis 

y„ - geometry: type of approximation ( 'line' , 'plane') 

% - visualization: figure Con', 'off') — default is 'on' 

% - sod: show orthogonal distances Con', 'off') — default is 'on' 

y. 

% Return parameters : 

% - Err: error of approximation - sum of orthogonal distances 
% - N: normal vector for plane, direction vector for line 
% - P: point on plane or line in 3D space 

1 

% Example: 
% 

% » XD = [4.8 6.7 6.2 6.2 4.1 1.9 2.0]'; 
% » YD = [13.4 9.9 5.8 6.1 6.7 10.6 11.5]'; 
y. » ZD = [13.7 13.1 11.3 11.8 12.5 16.2 18.5]'; 
% » f it_3D_data(XD,YD,ZD, 'line' , 'on' , 'on') ; 
% » fit_3D_data(XD,YD,ZD, 'plane' , 'on' , 'off ') ; 

y. 

y. Note: Written for Matlab 7.0 (R14) with Statistics Toolbox 

y. 

% We sincerely thank Peter Perkins, the author of the demo, 
% and John D'Errico for their comments. 

y. 

y. Ivo Petras, Igor Podlubny, May 2006 

% (ivo . petrasOtuke . sk , igor . podlubnyQtuke . sk) 
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